[Data Views] Disable scripted field creation in the Data Views management page #202250

davismcphee · 2024-11-29T02:23:09Z

Summary

Scripted fields in data views have been deprecated since 7.13, and superseded by runtime fields.

The ability to create new scripted fields has been removed from the Data Views management page in 9.0. Existing scripted fields can still be edited or deleted, and the creation UI can be accessed by navigating directly to /app/management/kibana/dataViews/dataView/{dataViewId}/create-field, but it is recommended to migrate to runtime fields or ES|QL instead to prepare for removal.

Additionally, the scripted fields entry in the Upgrade Assistant has been updated to reflect these changes, improve migration instructions, and surface the full list of data views across all spaces that contain scripted fields as well as their associated spaces. New documentation has been added to the "Manage data views" page with examples and instructions to help users migrate off scripted fields to runtime fields or ES|QL, which is also linked to from the Upgrade Assistant entry.

Data Views management page:

Upgrade Assistant:

Resolves #182067.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

…nal test

… deprecation

Bamieh

Markdown support for UA deprecations LGTM. Great addition!

wajihaparvez · 2024-12-04T00:31:06Z

@davismcphee Thank you for updating the docs 😄 @florent-leborgne is on paternity leave, so I'm reviewing this instead. All docs changes LGTM! Just suggesting some small changes for clarity.

docs/management/manage-data-views.asciidoc

docs/upgrade-notes.asciidoc

pheyos

test/tsconfig.json changes LGTM

sabarasaba

UA changes lgtm, left one small question

sabarasaba · 2024-12-04T09:29:10Z

...e_assistant/public/application/components/kibana_deprecations/deprecation_details_flyout.tsx

+            const textContent =
+              typeof currentMessage === 'string' ? currentMessage : currentMessage.content;


Is it possible that we might run into issues if currentMessage.content doesn't exist?

I think we should be ok because the only valid types are string or { type: 'text' | 'markdown'; content: string; }, and we account for each. Someone could do an unsafe cast when registering their deprecation, but that's true for most TS code.

kertal · 2024-12-04T12:09:53Z

src/plugins/data_views/server/deprecations/scripted_fields.ts

+      numberOfDataViewsWithScriptedFields: dataViewsWithScriptedFields.length,
+      runtimeFieldsLink: docLinks.indexPatterns.runtimeFields,
+      esqlLink: docLinks.query.queryESQL,
+      dataViewsList: dataViewsWithScriptedFields.map(buildDataViewsListEntry).join('\n'),


This is looking great, tested and works as expected, no scripted fields in a data view, no tab, just the manual way using the old URL to create a scripted field, which then shows the tab with scripted fields, and no button 🥳

One question, about the deprecation message, previously we showed just a max of 3 data views, but it seems we show them all now? or do I miss something. ... just for the case when there are many data views with scripted fields

We used to only show up to three titles in the message, but included the full list in the corrective actions:

kibana/src/plugins/data_views/server/deprecations/scripted_fields.ts

Lines 64 to 72 in cdb5a2d

i18n.translate('dataViews.deprecations.scriptedFields.manualStepTwoMessage', {

defaultMessage:

'Update {numberOfIndexPatternsWithScriptedFields} data views that have scripted fields to use runtime fields instead. In most cases, to migrate existing scripts, you will need to change "return <value>;" to "emit(<value>);". Data views with at least one scripted field: {allTitles}',

values: {

allTitles: indexPatternTitles.join('; '),

numberOfIndexPatternsWithScriptedFields: indexPatternsWithScriptedFields.length,

},

ignoreTag: true,

}),

I removed the ones from correctiveActions and instead put a formatted list in the main message. This could get really long for customers who have many data views with scripted fields, but the alternative is not having access to a full list, so I'm not sure what else we could do here.

Ah, that's how it works, thx, what else we can do? Nothing. Approving 👍

davismcphee

@wajihaparvez Thanks for the review and heads up about @florent-leborgne! Applied all of your feedback here: 8334e46.

davismcphee · 2024-12-05T01:11:04Z

...e_assistant/public/application/components/kibana_deprecations/deprecation_details_flyout.tsx

+            const textContent =
+              typeof currentMessage === 'string' ? currentMessage : currentMessage.content;


I think we should be ok because the only valid types are string or { type: 'text' | 'markdown'; content: string; }, and we account for each. Someone could do an unsafe cast when registering their deprecation, but that's true for most TS code.

davismcphee · 2024-12-05T01:19:35Z

src/plugins/data_views/server/deprecations/scripted_fields.ts

+      numberOfDataViewsWithScriptedFields: dataViewsWithScriptedFields.length,
+      runtimeFieldsLink: docLinks.indexPatterns.runtimeFields,
+      esqlLink: docLinks.query.queryESQL,
+      dataViewsList: dataViewsWithScriptedFields.map(buildDataViewsListEntry).join('\n'),


We used to only show up to three titles in the message, but included the full list in the corrective actions:

kibana/src/plugins/data_views/server/deprecations/scripted_fields.ts

Lines 64 to 72 in cdb5a2d

i18n.translate('dataViews.deprecations.scriptedFields.manualStepTwoMessage', {

defaultMessage:

'Update {numberOfIndexPatternsWithScriptedFields} data views that have scripted fields to use runtime fields instead. In most cases, to migrate existing scripts, you will need to change "return <value>;" to "emit(<value>);". Data views with at least one scripted field: {allTitles}',

values: {

allTitles: indexPatternTitles.join('; '),

numberOfIndexPatternsWithScriptedFields: indexPatternsWithScriptedFields.length,

},

ignoreTag: true,

}),

I removed the ones from correctiveActions and instead put a formatted list in the main message. This could get really long for customers who have many data views with scripted fields, but the alternative is not having access to a full list, so I'm not sure what else we could do here.

kertal

LGTM, tested and works as expected, no scripted fields in a data view, no tab, just the manual way using the old URL to create a scripted field, which then shows the tab with scripted fields, and no button. thx for moving scripted fields another step into there well deserved retirement 🦕

elasticmachine · 2024-12-05T21:38:29Z

💚 Build Succeeded

Buildkite Build
Commit: cf2ff64

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/core-deprecations-common`	12	15	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiAssistantManagementSelection`	93.0KB	93.1KB	+84.0B
`dataViewManagement`	140.1KB	140.1KB	+65.0B
`lists`	145.4KB	145.5KB	+84.0B
`upgradeAssistant`	159.8KB	160.2KB	+426.0B
total			+659.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	449.8KB	449.9KB	+84.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/core-deprecations-common`	20	23	+3

References to deprecated APIs

id	before	after	diff
`dataViewManagement`	35	40	+5

History

💚 Build #257197 succeeded 8334e46
💚 Build #256690 succeeded 0ab2795
💚 Build #256644 succeeded 367119a
💚 Build #256295 succeeded 9e258da
💔 Build #256292 failed 7b0984c
💔 Build #255791 failed 0c06115

cc @davismcphee

…ment page (elastic#202250) ## Summary Scripted fields in data views have been deprecated since 7.13, and superseded by runtime fields. The ability to create new scripted fields has been removed from the Data Views management page in 9.0. Existing scripted fields can still be edited or deleted, and the creation UI can be accessed by navigating directly to `/app/management/kibana/dataViews/dataView/{dataViewId}/create-field`, but it is recommended to migrate to runtime fields or ES|QL instead to prepare for removal. Additionally, the scripted fields entry in the Upgrade Assistant has been updated to reflect these changes, improve migration instructions, and surface the full list of data views across all spaces that contain scripted fields as well as their associated spaces. New documentation has been added to the "Manage data views" page with examples and instructions to help users migrate off scripted fields to runtime fields or ES|QL, which is also linked to from the Upgrade Assistant entry. Data Views management page: ![management](https://github.com/user-attachments/assets/ed42310c-d7fa-48ba-8430-533e9230b48d) Upgrade Assistant: <img src="https://github.com/user-attachments/assets/4ff27866-4fe8-4357-82e8-cd4c503ab50b" width="500" /> Resolves elastic#182067. ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [x] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Matt Kime <[email protected]> Co-authored-by: kibanamachine <[email protected]>

mattkime added 14 commits August 15, 2024 10:19

disable scripted field creation

1af7942

Merge branch 'main' into disable_scripted_field_creation

5bb4dce

remove button, add callout, update functional tests

bb2ac7d

remove unused localization

c396745

test fixes

3f3dfe8

test fixes

9b56eff

fix functional tests and Ãsnapshöts

4de4413

add i18n

aa5130e

better fallback when scripted field tab isn't available - for functio…

b5dac4d

…nal test

more functional test fixes

65ac465

fix functional test

4647a32

remove period from title

b308106

update snapshot

909d9c3

Merge branch 'main' into disable_scripted_field_creation

c3bdd04

davismcphee added release_note:breaking backport:skip This commit does not require backporting Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Nov 29, 2024

davismcphee self-assigned this Nov 29, 2024

davismcphee and others added 7 commits November 28, 2024 22:23

Merge branch 'main' into davis_disable_scripted_field_creation

d3ad3cf

Update docs and UA copy

6ee7c25

Copy fixes

33d3d19

Remove modified translations

5037ad5

Add markdown support to ugprade assistant and updated scripted fields…

6b44a98

… deprecation

[CI] Auto-commit changed files from 'node scripts/notice'

662f41b

Adjust copy

0c06115

Bamieh approved these changes Dec 2, 2024

View reviewed changes

davismcphee mentioned this pull request Dec 2, 2024

[deprecations/dataviews]: Check in all spaces #202354

Merged

davismcphee added 3 commits December 2, 2024 22:47

Merge branch 'main' into davis_disable_scripted_field_creation

b66b159

Update docs

c937a94

Fix broken test

9030958

Show data view name when available in UA

0ab2795